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3. Evidence 

The existing approved tests used by the -TOT's are the district wide tests, the 
P^^^ and ICRT, 
9. Method Criterion 5 Page 3 

All laboratories were visited at least three times. 
Evidence Criterion 1 Page 4 ' 

It is felt that in the one school where conditions were cramped fewer students 
than could normally be served had to be scheduled. Specifically, the Instruc- 
tional Aide could serve anywhere from 10 to 12 students, but in fact the size 
of the room dictated that no more than 8 be present at any one period. 
Criterion 4, Page S 

In one school the teacher sends students to labs but does not decide until 
Monday who shall go to the lab that week. This is considered inappropriate 
since the RENP plan for instruction of students specifies that there be 
sufficient time for the lA and TOT to plan together and this planning time 
be at least one week prior to the student's attending the lab. 
Criterion 5, Page 6 

Interviews were conducted with all lAS and TOTs . In addition, all principals 
were interviewed and 9 teachers were interviewed. Nineteen training sessions 
were observed. 
12, Criteria and Methods 

RENP is concerned that the math configuration program not merely be add-on 
or appendage to the existing school program, but that it be fully integrated 
with the mathematics programs at each school. Thus, our data collection 
methods focused on interviewing TOTS teachers, principals and specialists 
to determine the nature of the RENP mathematics program in relation to existing 
school program. We did not systematically collect data on the nature of the 
existing school -programs. We did seek to determine if the RENP math staff 
was instructing teachers to use new methods with their o\m curriculum and 
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if there existed conflicts on the services among the specialists. The 
coordination of servi%:e5 of the various programs is the responsibility of the 
principal as instructional leader; five of the principals appear from our 
data to be active in such coordination* 

13, Reviewers requested three additional pieces of inform^Ltion in relation to this, 
grant term: 

1. Adequacy of Procedures - Ife believe thar, the procedures used to assess 
employees were entirely satisfactor)". We based this conclusion on the following: 

a. Those aspects of employees' jobs which were as5essed were discrete, had 
behavioral. correlates, and were understood by the ewployees to be the basis for 
their evaluation. 

b. The instruments which were utilize.1 employed both observational field 
note data, and quantifiable judgemental data on a series of continuums. The 
information contained within the assessment forms vras readily accessible, easy 
to analyze and understand, and was directly related to the behaviors being 
analyzed, 

c Because the evaluation is an on-going process, it is sensitive to 
changes in behaviors, over time. 

2. How Many Interviews? - The number of interviews conducted in relation to 
this grant term and condition are as follows: All TOTS were interviewed; the 
COA was interviewed; all IAS were interviewed. 

3. Number of Teacher aiid Principal Interviews - Ten principals were interviewed, 
and nine teachers were interviewed, regarding TOT and lA performance. 

15. Method Criterion 4 

Grant term 16 describes how a 2S% sample or 370 students were originally 
selected to have their profiles analyzed. Additionally, 10% of this group 
had their learning activities compared with plans to determine the appropriate- 
ness of the learning activities themselves. 



16. Criterion 1 

States that almost all children will be .ihvolved in their math programs. Further 
evidence for this is cited in Grant Term 9, Evidence Criteria 4 and 5 
found on pages S and 6 of the report. 
19. The reviewer asked for five additional pieces of information in relation to 
this grant term, these are as follows: 

1. Number of TOTS Interviewed - All TOTS were interviewed. 

2. Time Lines - The same comments made in relation to grant term 29 are 
appropriate here. That is, the time lines are retroactive to January, and 
reflect to a certain extent, activities already under way. In our view they 
are realistic. 

3. Interviews of Unit Task Force Members - The following unit task force 
members were interviewed: All principals; all TOTS; ail COS; and all aides. ' 

4. Periodic Review of Student and Teacher Profiles - The same comments made 
in relation to grant term 29 also obtained in this case. That is: 

a. Periodically - There is no indication in most of the unit task force 
plans how often these profiles would be reviewed. 

b. There is no indication in the unit task, force plars regarding the 
number of such profiles that would be reviewed. 

5. What does the term "remarkable" mean as used on page 40? -'The term 
remarkable as used in this context indicates that there was a very high degree 
of congruent e between the interview data collected from TOTS and COS regarding 
the monitoring performed by the principal and the principal's delineation of 
her/his monitoring activities. 

21. Method 

At each Reading Center each of the TOTS were interviewed concerning the kinds 
of tests that were administered on three occasions. The diagnostic test data 
were reviewed by the evaluators to determine which specific tests were given and - 
all TOTS were interviewed to determine for what purpose those tests were given. 



5 



Evidence 

In clarification of the term "approved test'» it should be noted that the 
component head in Reading had approved a specific group of tests for use by 
the TOTS, Those tests are listed in the report. The TOTS were not allowed 
to use other tests than these cited without obtaining the peimission of the 
component head. 

The reviewers requested three additional pieces of information regarding this 
grant term as follows: 

1, Adequacy of Procedures - Tne conaaents which we made in relation to this 
point for grant term 13 also hold here, as the procedures used on reading and 
math are virtually identical. 

2, Number of Interviews - The following number of interviews were conducted!* 
All TOTS; both TOAS; and 20 IAS were interviewed. 

3, Number of Interviews with Teachers and Principals - 25 teachers were 
interviewed, all 10 principals were interviewed. 

Method Criterion 4 

The records on teacher training of each TOT for all of their teachers were 
scrutinized. The data collection plan for formative evaluation included the 
desire and intention of observing systematically a representative group of teach- 
ers for each TOT who had completed training in certain areas wjiile recognising 
the sensitive nature of such systematic observation by evaluators. In our 
initial development of a data collection plan, the consensus was reached that 
such systematic observation of teachers would only be undertaken when such 
would not jeopardize the working relationship between the TOT and the teacher. 
It was hoped that sufficient time and contact between the evaluators and teachers 
in schools would allow access to teachers' classrooms for this purpose. Such 
systematic observation was begun and completed on only four teachers in reading. 
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We acknowledge that this is an extremely small number relative to the 115 
teachers being trained in reading. As mentioned in the section entitled 
"Has the Grant Term Been Met? Criterion 4" we further acknowledge the limited 
generalizability of this evidence in the area of teacher con^ietency. The 
documentary data cited in this section on page 48 refers to the TOTS* records 
of teacher training. 
Evidence Criterion l,^Page 49 

All of the TOTS were interviewed and all understood these competencies were the 
basis cf the teacher training program. 
Evidence Criterion 3, Page 51 

The sufficient time allowed by the TOTS for debriefing is meant to convey that 
the TOTS did allow at least 10 to 15 minutes at the end of each demonstration 
lesson to discuss with the teachers the purpose and effectiveness of the 
lesson. 

Criterion 4, Page 52 

As was previously discussed not only were TOTS' records of teacher training 
scrutinized, but observations were initiated for the systematic evaluatiga of 
teacher competency. 
24. Criteria an d Methods 

A particular concern of REN? is that the reading configuration program not 
merely be an add-on or appendage to the existing school program, but that it 
be fully integrated with reading programs at each school, llius, our data 
collection methods focused on interviewing TOTS, teachers, principals and 
specialists to determine the nature of the RENP reading program in relation 
to existing school programs as perceived by these people. While we do not 
have extensive data on the nature of the existing curriculum in each school 
and the Title I program in each school, we have examined the process by which 
the Component Head the the TOTS have worked with school personnel to 
initiate the RENP program in their schools. As cited in Evidence Criterion 1, 
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the Component Head met with the District Coordinators and with the school 
Priljicipals. All TOTS met with principals at their schools also. We have 
verified these meetings through our discussions with each school principal. 
As the instructional leader of the school, the School Principal has the 
responsibility for the coordination of the services of each special program 
within the school. From our interviews with principals, it was determined 
that only 5 of tlxe 10-^ principals of schools with reading components in 
place do spend considerable amount of their time engaged in the coordination 
of instruction or instructional programs or the monitoring of the REMP program. 
This was suggested in grant term 31. Thus, it would seem that In the other 
five schools, the coordination of the instructional programs is left mainly 
to the staff. The integration of RENP with other programs at these schools 
is wholly dependent upon the active involvment of the TOT in each school with 
the teachers and specialists in that school. In the five schools with 
principals active in program coordination^ TOTS working with the school staff 
can be more efficacious since the burden of instructional leadership is 
borne by the principal. Our interviews with teachers, specialists and princi- 
pals and assistant principals show, quite clearly that the TOTS in all 10 schools 
have taken direct action and initiative to meet weekly with the specialists 
involved in an attempt to coordinate the services cf these programs. 
H as The Grant Term Been Met?, Criterion 1 

The lack of specificity stated in the report refers to the fact that although 
the plan stated what was to be done, how it was to be done was not specified. 
Criterion 2 

IVhen it was stated in the report that the TOTS have generated the necessary 
specificity in the operationalization of the plan, this was meant to convey 
that while the plan did not thoroughly specify how the meshing was to occur, 
in practice the component head has developed with the TOTS in their staff 
development sessions specific ways by which ;the TOTS may tailor thf: meshing 



of children's skill development and functional applications to the individual 
teachers in that school. Thus, while there are generalizable procediares being 
followed, each TOT does have individual latitude in developing programs for 
each teacher in training to mesh skill development and application, 
26, Method Criterion 4 

The way in which the two sampling procedures were performed is described in 
Grant Term 27, wherein the students were partitioned into two categories and 
25% of students' records and plans for learning activities were analyzed for 
each category. 

In addition, a 10% sample of this group of student records were- analyzed by 
comparing activities to objectives. Thus, 780 student's records were initially 
analyzed in terms of learning activities, and over 80 were analyzed thoroughly 
by comparing activities to objectives on tw:> occasions ^ encompassing all ten 
schools. 

Criterion 1, Page 61: Erratum^POT should read PRT, 
Method Criterion 4 

The analysis to deteimine the appropriateness of activities to plans entailed 
CI) reviewing the plans which were based on the diagnostic tests data and 
(2) reviewing and observing the learning activities to determine if they did 
address the specific objectives of the learning plans. If those activities 
did directly address those objectives the activity was deemed appropriate, 
7. In response to the Project Officer's concern that students be observed and the 
data reported, as was mentioned under Grant Teim 30, on three occasions each 
center was observed to determine the nature of the instruction being provided 
to students. On each of those occasions the instruction being provided the 
students by the aides was observed and individual profiles and plans of 
several of the instructed students were examined to determine the appropriateness 
of the learning activity being undertaken. In all cases the evaluators deemed 
that the instruction being provided was appropriate for the diagnosis. 



28, Method 

The basic observational st:rategy for the Reading Centers in this grant term is 
described in Grant Tern 30. The centers were observed to detetmne the adequacy 
of facilities and materials and the centers' usage hy duldrer and teachers. 
All TOTS were interviewed to explain the physical set-up and schedule of 
services and how the materials were being used. .Activities on-going during 
the observational period were crossed checked with the planned scheduled 
activities for each center durij^g that time period. 
Bvidence 

The proposed central teaching center in the R£NP Continuation Proposal was to 
be a center wherein teachers from the region would come to receive instructional 
improvement through sessions with TOTS. That central teachers » center is not 
operational, however, at McZogney Annex there is a central resource center for 
instructional materials which the TOTS use. 
29. The reviewers wished five additional pieces of information in relation to this 
grant term, these were as follows: 

1^ Number of TOTS Interviewed - All TOTS were interviewed • 

2, We reported one of the common project objectives that recurred across unit 
task force plans. Two questions were raised by the reviewer. : 

a. In the hypothetical instance cited by the reviewer ''does this mean that 
the student is only required to master five?" It is our understanding that this 
is the case, since 50% mastery is the criterion used for this grant term. 

b. Why is the ICRT used? - The statement which we made in our report is 
in error, for "ICRT" you should read "PRP*. 

3. Time Lines - In our view, the January to May time line is realistic in that, 
even though the plans were siibmitted late, the activities did begin in January, 
Thus in part, the plans were retroactive^. Further, it is our belief that 

the plans were careful to Si^ipulate only activities which had a realistic 
chance of occurring during those five months. — , ' • ' 
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4. Interviews with Unit Task Force Members - The following numbers of unit 
task force mOTbers were interviewed: All principals; and TOTS; all COS; and 20 
aides, 

5. Period ic Review of Student and Teacher Profiles - The reviewer raised two 
points for our response regarding this aspect of the grant term and condition. 

a. What is meant by periodically? - Most of the reports did not stipulate 
what they meant by periodic review. One of the reports however, indicated that 
such profiles would be reviewed monthly. 

b. Number of Student and Teacher Profiles to be Reviewed - Again, the 
reports did not stipulate a specific number of student and teacher profiles 
which would be reviewed. 

30. Evidence, Criterion S, Page 76 

The examination of profiles and plans refers to the analysis and examination 
performed in Grant Term 27. The IAS and TOTS interviewed refer to all TOTS 
and 20 IAS. . Further, all TOTS and principals were interviewed as well as 
25 teachers. 

32. Method Criterion 2 

It is stated that the director was interviewed regularly, this should be 
interpreted to mean weekly. s. 
Criterion 3 

Two meetings were held to review, devise and modify the specific milestones 
in monitoring criteria. Both meetings were attended by the evaluator, three 
members of the ACSB were in attendance at both meetings and they reviewed 
the finalized milestones and the evaluation methods. This information is also 
relevant to Evidence Criterion 3, page 82. These members of the ACSB were 
considered appropriate as they were the chairpersons of three committees. 
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Method Criterion 3^ Page 87 

All 10 Community Organizers, two Senior Community Organizers, and three 
staff members of the Information Dissemination and Resource Center were 
interviewed. 
Criterion 4 

All 10 Community Organizers and two Senior Community Organizers were 
interviewed. In addition, four randomly chosen Community Organizers were 
selected to be observed in their daily activities for two consecutive days- 
During that time, the activities in which they were engaged were recorded 
and compared to their job descriptions to ascertain if they were in fact 
performing duties in accordance with their roles." 
Criterion 6 

All weekly activity reports of each Comnnmlty Organizci- were reviewed. All 
10 Community Organizers and two Senior Community Organizers were interviewed. 
Evidence Criterion 1, Page 88 

Personnel folders of members of the PCI staff were reviewed. Interviews were 
conducted with all Community Organizers, two Senior Coimunity Organizers, and 
three staff members of the IDRC- 
Criterion 3 

Interview results indicated that 13 of the 15 staff members interviewed felt 
that they previous training they had received did provide appropriate skills. 
In addition,. 10 of the IS interviewed indicated a desire for trairdng in 
report witing, these were the 10 Community Organizers. 
Criterion 5, Page 89 

All IS principals were interviewed. In addition, two principals, two Local 
School Board Chairpersons, 8 School Board members and three parents were 
interviewed. The parents indicated that the CO was well knoim to them and 
they were well aware of her operation in -the school. That is. they could 
cite specific examples of activities the CO was engaged in and noted frequent 
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contact with the CO, 
Method Criterion 1 

The phrase "requisite elements" refers to those items listed under Critera 1. 
Method Criterion 1 

The RENP officials who were interviewed include the Director, the Public 
Information Officer, and the PCI staff memb^^ jp the IDRC. 

Criterion 3 

There are 3 IDRC staff members. 

\ 

• EYj.dLence Criterion 1, Page 97 
The local residents interviewed concerning the IDRC services were five people 
who used the referral service during a one week period in April. • 
Criterion 2 

The PCI officials maintain that the unique function provided by RENP'S IDRC is 
that it is a conduit for referrals of information for service to other community 
agencies and as such, anyone may use it to get information on problems that are 
related to any agency in the area. The PCI officials referred to in Criterion 2 
are the Director of RENP, the Public Information Officer and the staff member 
responsible for the IDRC. 

The reviewers sought clarification of the following three points: 

1. What does, the term "PCI took pains" mean? - As used in this context, this 
term means that the developers of the seven plans to which reference is made 
in this statement were very careful to relate PCI objectives to overall 
project-related objectives. That is, the developers of these seven plans made 
sure that the support objectives delineated for PCI would facilitate the accom« 
plishment of project objectives in general. 

2. IVhat does the term "exemplary" as used on page 100 mean? - The term 
exemplary as used in this context, refers to those seven unit task force plans 
which had carefixlly delineated activities and objectives for the PCI component. 



These can be contrasted to the other schools which, while they did have some 
activities delineated for PCI, were not- nearly as specific as those to which 
we referred as exemplary, 

3« What does the term "frequent** as used on page 102 mean? - As used in 
this context, the term frequent should be construed as meaning that the 
coraniunity organizers make regular visits to homes. In our view, most of 
these visits occiir on- an as-needed basis, and the actual number of such 
visits varies in proportion to that need. 
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Introduction 



The purpose of this report is to provide conclusions and evidence relating to 
the implementation of certain grant terms and conditions negotiated between the 
National Institute of Education and the Response to Educational Needs Project 
which were finalized on February 13, 1976, The conclusions and evidence delineated 
in this document pertain to grant terms and condition numbers eight through thirty- 
seven, and focus upon the Reading, Mathematics and Parent/Community Involvement 
components of the project. Terms and conditions pertaining to the Management 
component of the project are not addressed herein. Most of the data on which 
claims made in this report are based were collected between February 23 and 
April 23, 1976. Data were collected pursuant to the "Formative Evaluation 
Design/RENP", developed by the evaluators and modified through inputs provided 
from the National Institute of Education and the Response to Educational Needs 
Project • 

Cantents of the Report: 

T&e remainder of this document is divided irosD -nree sections: a summmary 
of the extent and quality of implementation ceE grant term and condition; a 

lengthy explication of extent and qualit^r of the implementation of each 
"grant term and conditicECL including concomi* t^arc cr;rrreria and methods used 
:for evaluation; and finally, a brief sectiom y^ich attempts :t:o cross-cut 
the grant :^rms and conditions and to summsi-jriz^ conclusions Telative to the 
number of schools in which RENP is fully or vmrtX^lly implemented. The bulk o^r 
this document is comprised of the middle se^ttion which speaks to the degree and 
quality of implementation. The format utilijied ±a organizing that section is 
presented below: 

o Grant Term and Condition: Each grant tenro amd ::ondition is considered 
separately. This section repeats the grant cetms and conditions verbatim; 
as they appeared on the final list negotfeiterf between the Institute and the 
project. Each grant term and condition is mannered according to its placement 
on that list. ^ ^ 



. ° Sndtd; iTr t^JS^^^:^'^ "^-'^^ "^^^^ -xe used as • " 

^IfnJ''^ Grant Term Been Met?: This subsection, which appears for each 
Sch of'SIe'criJS''''?"' ^-""^-"^ ^he degree of in^plLentatlon ?o1 
eact c^iJerioJ Js ^^^^'^'^ °^ implementation of 

not can Si ^ • '^^f^^^^d. Some of the criteria however, do 

not call for qualitative judgments. In such cases only the extent of 
. implementation is discussed. ^ extent of 

o Evidence: This section, which is included for each grant term and * 

sSfja;°t?;t:TTa- J'^'^^^ ^° SKerLTwSch 

Poss?Me the ivT subsection described above. IVherever 

givei da^a sets n ^^^P^^^^^^ appropriate numbers regarding 
"X" was Jhe"^' f ^T- P^^""^^g« °f teachers contended that 

A was the case, or a certain phenomonon was observed in "X" percent of 
the schools) . Names of specific schools and/or individuals are^no^c^ted. 

The reader will note, in perusing this report, that for many of the grant 
terms and conditions there are several criteria relating to extent of implementation 
and degree of quality. When multiple criteria are involved, the evidence 
often reflects "mixed" results for a given grant term. For example, if there 
are four criteria for a certain grant term, it may be that: criterion one has 
• been met with high quality at all schools; that criterion two has been 
partially met, with varying degrees of quality at all schools; that criterion 
three has been met with high quality for some schools and has not been met 
at all in others; and that criterion four has not been met at any school. The 
evaluators have made no attempt to weight the importance of the criteria in 
relation to each other. Therefore, in the case of "mixed" results, we leave it 
to the reader to decide if the number of criteria which have been met, and the 
degree and quality of their implementation is acceptable. 
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SUMMARY OF TBE EXTENT AND 
IJUALITY OF EACH GRMT TERM AND GDNDITION 
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MATH COMPONENT 

8. Use existing diagnostic instruments 

One criterion was used and was fully met. Therefore, the grant term was 
. fully met, 

9, Stabilized program at 10 schools for children 
Give criteria were used. Two were fully met, and the remaining three 
were partially met (fully met at nine of ten schools, another criterioii WUb 
£u^^- sssf^a of ten sckaals and tke remaining criterion was fully met 
at eight of tefn schools). Thus, the grant term has keen partially met. Two 
of these criteria necessitated, qualitative judgements.. One was fuUy 
met with high quality. The renaaining one has been fuOIy met with high 
quality at eight of ten schools. 

10. Operational program at 10 schools for teachers ' 

Three criteria were used- Two cri^teria were fully net:, and one criterion was 
partially met (fully met at eight of ten schools). The grant term, therefore 
was partially met. One criterion necessitated qualitatir/e judgement; it 
was fully met wixh high quality. 

11* Man to mesh skill development and application 

Two criteria were used. Both of these criteria necessitated a qualitative 
judgement: both have been fully met with high quality. Thus, the grant term 
has been fully met. 

12, Plan to integrate .HENP with .schools 

Two criteria were .used. Both necessitat^ed: qualitative judgements and were 
fully met with hi^ quality; the grant term, therefore, has been fully met. 



Assessment of staff competency - 

Five criteria were iised. All five necessitated qualitative judgements and 

have been fully met with high quality. Thus, the grant term has been fully 

met. 

Assessment of teachers 
Four criteria were used. All 'four necessitated qualitative judge:nents: 
V V ui*la 1. h^exi fully met with high quality, and two have been 

partially met with, high quality (fully met at seven of ten schools; the 
other has been partially met and reflected by limted data}* As a whole, 
the grant term lias been partially met. 

Assessment of students 

Four criteria wes^^ identified. Three of these criteria have been fully met 
and one critericc: was been partially met (fully met at eight of 
ten schools). Cbrarall, the grant term has been partially met. 

Most children accommodated 

Two criteria were used. Both criteria necessitated qualitative judgements: 
both were fully met with high quality at nine of ten schools. Therefore, 
the grant term was partially met. 

Focus on enabling objectives 

One criterion was used and was fully met; the grant term has been fully 
met. 

Use existing materials 

One criterion was used and was fully met; the grant term was fully met. 
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19* Unit tacic force plans 

Six criteria were employed. Five criteria necessitated qualitative 
judgements: all have been fully met with high quality. in addition, 
the remaining criterion has been fully met; me grant term has 

been fully met. 

READING COMPONENT 

2(K. No new curriculum development 

One criterion was used and was fully met; the grant term was fully met. 

2L. Use existing diagnostic instxuments 

One criterion was used and was fully met; the grant term was fully met. 

22. Assessment of staff competency 

Five criteria were used. All necessitated qualitative judgements and have 
been fully met with high quality. Therefore, the grant term has ' 
been fully met. 

23. - Assessment of teachers 

Four criteria were used. All necessitated qualitative judgements : two were 
fully met with high quality and two were partially met with high quality 

(fully met at eight of ten schools; the other has been partially met as 

J 

reflected by limited data). Thus, the grant 1term, as a whole, was 
partially met. 

24 Plan to integrate RENP with schools 

Two criteria were used. Both necessitated qualitative ju%ement:s: both 
were fully met with high quality. Therefore, the grant term was fully met. 

25. Plan to mesh skill development and application 

Two criteria were used, and both necessitated qualitative judgements.. One 
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has been met, but not at high quality. The other one has been fully met 
with high quality. Hence, the grant term, as a whole, has been 'partially 
met. 

26. Assessment of students 

Four criteria were used: three have been fully met and one has been 
partially met (fully met at ::3even of ten schools). Therefore, the grant 

* 

term , as a whole, has been rpartial ly met. 

27. All children accommodated 

Two criteria were used. Both required qualitative judgements and have been 
fully met with high quality. Therefore, the grant term has been fully met. 

28. Focus on enabling objectives 

A single criterion was used snd was fully met. Therefore, the grant term 
has been fully met. 

29. . Unit task force plans 

Six criteria were. used and were fully met. Thus, the grant term was fully met. 

30. Stabilized program at 10 schools for children 

Five criteria were used. Oner criterion has been partially met (fully met at 
seven of ten schools). The remaining four criteria necessitated qualitative 
judgements: two have been fuUy met at high quality, one has been met but 
not at high quality, and the fourth has been partially met with high 
quality (fully met with high quality at seven of ten schools). Therefore, 
the grant term as a whole has been partially met, 
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Jl. * Operational program at 10 schools for teachers 

Three criteria were used. One criterion has been fully met, and 
another has been partially met (fully met at seven of ten schools^. The 
remaining criterion necessitated a qualitative judgement: it has been 
•fully met with high quality. The grant term, as a whole, has been partially 
met. 

PARENT/COMMUNITY COMPONENT 

32.. Assistant Director; ACSB and local school board milestones 

Five criteria were utilized. Two criteria have been fully met, one 

has not been met, and two have been partially met. We conclude: that the 

grant term in general has been partially met. 

33. Assessment of staff 

Six criteria were enumerated. Five of these criteria have been fully met, 
and one has been partially met. '''^ find, therefore, that overall the 
grant term has been partially met. 

34. ACSB and local school board competency profiles 

Two criteria were used. Neither of . these criteria have been met and therefore 
the grant term has not been met. 

35. .Businesses, agencies and institutions 

Four criteria were used. Two have been fully met and two have been partially 
met. Therefore, the grant term, as a whole, has been partially met, 

36. Community referral service 

Two criteria were utilized. Both criteria have been fully met, therefore, 
.. the grant term has been fully met. 
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Unit task force plans 

Two criteria were used and both necessitated qualitative judgements. 
These criteria have been fully met with high quality by seven schools 
have been fully met with lesser quality by seven schools, and has not 
been met by one school. Thus, in gneral, the grant term has 
been partially met- 



EXPLICATION AND EXAMINATION 
OF- GRANT TERMS AND CONDITIONS 
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. RENP will select and use existing diagnostic instruments in mathemetics 
rather than developing new ones. Informal tests designed to facilitate 
day-to-day classroom work will be permitted. 



Criterion 

• TOTs will use' only existing diagnostic tests. 
Method 

To determine if the criterion has been met, inspections of diagnostic test 
data and interviews of TOTs were conducted. 

Has the Grant Term Been Met ? 

This grant term has been fully met. 

Evidence 

Existing approved tests are used by all TOTs. These include the ICRT and PMT. In 
addition, one TOT used the standardized Mathematics Level Test. Also, sections of 
certain other standardized tests are used for checking mastery of skills by students. 
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9, Implementation for the mathematics component will begin on January 1, 1976 with 
the understanding that all 10 schools stated in the continuation proposal will 
have in place high quality, stabilized program servicing children. This condi- 
tion will constitute consideration at the April ,26, 1976 program assessment. 

Criteria 

. 1, Space and facilities for laboratories provided by schools will be adequate, 

2. The labs will be fully stocked with instructional material. 

3. The materials will be cross-referenced to appropriate objectives, 

4. Children from each teacher's class will be scheduled for instruction in 
the laboratory and served in accordance with the schedule. - 

5. Instruction of students in the lab by lAs and, in the classroom by TOTs will 
be of high quality: 

- lAs' instruction will reinforce skills for appropriate objectives, 
evaluate learning, and record mastery, 

- TOTs will plan for lab instruction by lAs, coordinate lab and classroom 
instruction with teachers, plan and execute with teachers classroom 

. instruction to introduce or reinforce skills, evaluate learning, record 
mastery and coordinate all student learning and mastery with teachers. 

Method 

Criteria (1-4) All laboratories were observed at least three times (once per 
month) for evidence of the adequacy of facilities, and materials, the cross 
referencing of materials and each lab's usage by children. All TOTs and lAs were 
interviewed on these occasions to explain the- physical set-up and schedule of 
services. Materials and their cross-referencing were closely examined to 
ascertain the accuracy of cross-referencing and to form a general opinion of 
the quality of the materials in use. - • 
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Criterion (5) On each visit to labs the instruction of students by the lA was 
observed and on each occasion several students were "instructed'* (e.g., talked 
with concerning their lab assignment to ascertain their understanding of the 
material which should be of a reinforcing nature). After such contact with 
students their individual profiles and plans were examined (for further discus- 
sion of the profiles and plans see grant term 15) to determine the appropriateness 
of the learning activity and to form a general opinion of the accuracy of the 
diagnosis. The recording of student mastery was also noted. Further, the lA 
in each lab was interviewed concerning the specific instruction performed and ' 
the procedures for planning and executing instruction and for evaluating 
leaiTiing and recording mastery. 

On each visit to the lab we had at least a brief opportunity to talk to the 
TOTs to corroborate information from the lAs concerning the labs' operation. 
In addition, each TOT was observed in 19 training sessions with teachers 
which provided an opportunity to check on the process of coordinating instruc- 
tion and learning mastery with teachers. Also, eadi TOT was interviewed on 
several occasions concerning the total service program at each school. 

To provide corroborative evidence of the nature of the service to children at 
each school the principals and/or vice principals were interviewed. Also, at 
least one teacher working with each TOT was interviewed. In total, nine teachers 
ViTore inteorviewed, covering eight schools (two- thirds of the six TOTs split schools 
and for two of them we could not arrange an interview with a teacher at one of 
their schools). 

Has the Grant Term Been Met? 

Criterion (1) has been fully met in nine schools and not met in one school. 
Criterion (2) has been fully met. 
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Criterion (3) has been fully met at seven schools and partially met at three 
schools. 

Criterion \S) has h&sn fully met at eigsrt ::schools with high quality, met with 
lesser ta^;.irry at one school, and only-^srtnially met at lesser quaildty at one 
school. 

Crit xim (Sl has beer fully met at hir r q,ijar_ty in all schooLx 
Evidence 

Critericif In the ten schools we fosncL aat nine of the ten labs were 

operating ir classrooms provided by the 31-100I. In the othe:r school the lab 
is operati^r.g in a former storage room; conritions are cramped -and fewer students 
than the lA could serve must be scheduled as a result. 

Criterion (2) In all ten schools we found the labs to be fully stocked with 
materials, the range of which adequately covers the laboratory approach in the 
topic areas of the D.C.P.S. curriculum. Most of the materials come from Creative 
Teaching Associates and are already indexed to objectives by topic area. 

Criterion (3) The laboratory materials from Creative Teaching Associates are 
indexed to objectives which permit their effective use in each topic by TOT and 
teachers. Additional material in the labs did require indexing, however. Five TOTs 
had been shifted from their intended schools and found that they had indexed their 
materials to the wrong set of objectives (i.e., the new school had administered the 
PMT when the TOT had indexed materials to the I CRT or vice versa). Also some 
schools had administered both sets of tests to different groups of students. The 
PMT was constructed to correspond exactly to D.C.P.S. objectives, so that indexing 
to PMT skill levels is "tantamount to indexing to D.C.P.S. objectives. The ICRT' 
objectives do not follow the same sequence as the D.C.P.S. objectives. 
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•Ib February, when the fomrative evalustoi:^ be^am collection, TOTs were in 

the process of indexing materials to olrjact^^w- at- *2ach laboratory. The TOTs 
collaborated on a nnastery correlati-^n or JTC ann£ BIT oijectivtss v^-hich would 
perMt easy cross-referencing. This proc2::2 « verr;i: nearly completed.. The TUr^ 
in seven schools have all material indexe • iiii thhe trrcL.. c areas being addressed 
by teachers this year and additional materia. mdExs:- -o ICRT so that when the 
correlation is completed this cross- referencin. ^-iLlIL-uq straight for\s^ard. In the 
ot22ST three schools approximately 60% of the? t^. .1 by our sasnpling, is apprc- 
priatley indexed. 

Criterion (4) In eight of the schools studer .::!e ae^duled from the class of 
each RENP teacher into the lab on a regular -i^a is- niMl the teacirer, TOT, and lA 
know a week or more in advance which students ax:^ crarr^g on which days. Thus, 
they can select learning activities based upo: thessb students ' individual learning 
plans. In one school, although each teacher s^-Ty'fe ^students to the lab' daily, the 
-teacher decides on Monday who will go that weei; sand rhe lA soes not know who is 
coming until they appear. Planning for instxnc^don is done by the TOT and lA 
on Monday while the students engage in gaming xrr manipulative activities. We 
do not judge this scheduling procedure as apprapriats. In the remaining school, 
considerable scheduling confusion ensued for o^r a. rHnnxh when inot all teachers 
nor their students were regularly scheduled fox -rr.?- jaaiPOratory. 

Criterion (5) Our observation of the instruct±Ern2LL performance of the lAs 
indicated that all were reinforcing skills, not introducing new ones (the TOTs' 
or teachers' responsibility) through skillful. teaching activities. Iftiile there 
was some variability among aides, all were judged cinnpEtent and several to be 
extremely competent (in fact, their teaching wa^ as ^ood or betnter in the 
reinforcing activities than that of teachers). The liAs lappear. to be not aides 
in the usual sense, but highly trained special:.sts in a narrow range of instructional 
activities. g 
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From the interaction with students and me examination of their profiles and plans^ 
the -instruction provided wa:, deenssd appropriate. Also., in evidence were tests 
evaiiuating learning and records o-f masstery at each school. The procedures were 
uniformly carried out by the lAs. 

Intervier^'s were conducted with' lA^ and: TOTs. Data from the two sources are congruen- 
in indicating that lAs and TOTs plan tzpgether for the lAs' instruction. We never 
obsa-r^/ed any planning sessions, which usually occur after school, as it was this tine. 
thsE: was used for interviews. However, one lA was observed planning for her student5 
wkfin the TOT was absent; although, lAs are ostensibly not supposed to plan, this 
Ion obviously merited some role flexibility. 



Hie ^interaction of TOT and teachers in terms of coordinating learning activities 
and svaiuation was assessed via observing the training sessions and interviews 
with: TOTs, teachers, and principals. The demonstration lessons taught by TOTs 
were we 11- executed. The planning sessions observed did evidence the desired 
coprdination and interviews cooroborated that weekly or biweelcly coordinated 
evaluation occurs. Particularly in evidence was the updating of all test data; 
though tests were administered in September, teachers and TOTs systematically 
updated the data on levels of student learning for the Spring through teachers* 
analyses and check tests in the areas of the curriculum treated. 



t - 
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10- It is eacoected that all -^(yncr^^^ lar^c ar-.A ^ w , 

will be operational and lS^g^I^:'^ZiT2^'^'r^'''''l 
teachers in need of rrai^ing snoild havfcomple^e" tin ng'by A^f^e ' 

max aecrsioij point. The laanber af labs and centers v,: - ; be t&e aoEsber 
stated m the accepted proptJSEi. of December, 197S. 



Cri-teria 



1- The ten laboratories for feriem^cs training will be opeisrioEal 
and servxcdng teachers. 

2, Teacher trafaing scheduled-thrxnagh April 26, 1?76 shall ha^^ been 
completed. 

3. Teacher training completed shall ha^re addressed the areas of 
teachers' most n^ded skills from the RENP competence criteria in 
the teaching of mathematics. 



MetJiifc: 



. Criterion (1) The operation and ser^ce of rhe laboratories was assessed through 
the observation of 19 traizning sessions in the ten schools involving 13 teachers 
(or 15% of the 85 teachers). 

Criterion (2) The accomplishment of srhedulexr teacher training was ass^sed 
through the scrutiny of all TOTs- files on. eadh of their teachers and an 
examination of all TOTs- weekly reports on activities t:.- :the component .read. 

Criterion (3) The addressing of th^e^eas of teachez^^ .ost :.eeded skills was 
assessed though interviews with all TdTs. ^ princis^als or a^ant principals, 
and a sample of teachers randomly selected ^d stratified by ST (9 teato or 
11% of the 82 teachers). 
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Bgs the Grant Tearr Been Metl 
Crrtexion (1) has zeen fully iirct. 

Cr±terion (2) has ^een fully jnet in ii^t :^'ools and part±aiiy»^ net in 
tW33 schools. 

Cx±terion (3) hss hssn fully mex with a r.r^h degree of qualify. 
Evidence 

Criterion [1) Fror. reports ox the scnHfrr-T'e.d usage of centers ^or teacher 
training and studenr instruction, obs:.2rcariun days were randcndlv chosen. We 
found all ten centers to be servi'cing teaaiisrs^ One was suffer:rn:g under adverse 
conditions in terms of facilities, hut the txa±niirg proceeded navertheless. 

Criterion (2) From scrutinizing TOTs' fijies an each teacher amd weekly reports, we 
Tiave determined thas:, with the except±Dns of^rteaciier illness oir pregnancy or TOT 
illness, all teachers receiarad trainiii^ as scheduled im seven schools. In two 
schools two and three teachers hair^ not b-ee^i attending scheduled training sessions; 
in the remaining school, teachers we^^ impnmerly scheduled, in our opinion, so that 
some vould receive txaining on an :5~needed basis only. TS-s, at this school only 
those teachers whom the TOT beliesssd needed traiirmg wer.i ^^snred. Our undsrstanding 
of IRENP however.. Is that all taacfeisrs^ are tro receiiye ser^^dcj^ 

In relating crinerion (1] t& iz=± V^ Ljm 'CI^l, fioumd ttet ±ir all observed training 
sessions, the teachers who: w^ace ^ ffy r m aeci did atnand, winhrthEr^ceotions of those 
on leave. We diii find, jjn additisii,. :r?32E.i:nst:anc5r ^ah.^ ^ 3DT cancelled the day's 
training at one r^chool Xn order tcr 32ffisp:mi.lab OHem anik^T arther school when .the lab 
manager was ill- we soughx claxifiEEiiiian^ of RENP ^oliry prrcxxties and were told ' 
by the component head, that the trsnrrn^.of teachers has prioritrv' over the lab 
service to students and such canceilin^^of training should not^inave occured. No 
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other incident of this kxnd was observed. 

Criterion (5) In iheir -interviews the TOTs all stated tnac tb*^ negotiated 
draining plans reflected a synthesis of teachers' felt needs ^sujd TCTs' perception 
of teacher needs. Of note is that initially none of the tsaciiJ-rs very 
familiar with the laborarory approach or the KENP leaminr seqtisnxn-. Therefore, 
trsiniiig in mathems-tics was quite uniform in procedure yi±t:\ varaa .lens of tesacfcers' 
curricula and content lcnc;;flsdge. kll i\ine interviewed te^.cheTs sza^ed tha!t the 
TOT was working with them on what they considered to be tudsZ iincrartsnt areas. 
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ll. By January 15, a detailed plan for training teachers to mesh chdldrens' 
sKill development and application will be prepared; by March 1st the plan 
will be implemented. 



Criteria. 

1* Ai de*tailed plan for training teachers to mesh childrens' skill 
development and functional applications will have been prepared. 

2. The plan shall be ianplemented; TOTs shall be trainirrg teachers: 
• to diagnose individual student skills 

- to group students of similar needs for instrtiicrjLam 
to plan instruction for groups and individuiai= ^ased 
upon diagnosis 

- to teach skills within an applied context in tthe regular 
mathematics period and in the conteoit areas science 
and business)* 

Method 

1* The plan that was submitted was revieweai to detexiiEini?^ ££ it aizi 

tke meshing of skill development and applicaticsn in i^aadier msinrOT^ 

2* The implementation of the training plan was assessed r&rough i±ie 
interviewing of all TOTs and a sample o£'9 teachers of t3m S5 

teachers) and through observation of 19 training sa^ions. 

Has the Grant Term Mean Met? 

Crilrgyifion (1) has been fully met at high quality. 
CrrteioEDn (2) has beem fnlly met at high quality. 
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Evidence 

Criterion (1) The submitted plan describes the basic format of RENP teacher train- 
ing in mathematics. It contains ths five stage learning sequence which teachers 
will be trained to use via the labciratory approach with their students, A 
full explication of these stages vcas not contained in the report but was provided 
to the evaluators by RENP, reproduced from the reference cited in the report or 
plan. The process of teacher training was described in the plan, delineating 
training procedures for teachers in each of the five stages. The fifth stage 
of this sequence is Application (as stipulated in this grant term) in which the 
teacher specifically promotes the transfer of skill development to other contexts 
through "experience in applications in a variety of situations". 

Criterion (2) The TOTs are employing the five stage learning sequence in their 
training of teachers through the laboratory approach. In interviews, all six 
TOTs: expressed that it was iniportant: to help teachers appreciate the necessity 
'of moving beyond the fourth stage oxT "Fixing Skills" (the reinforcement of 
learning). In the eleven training «ssions observed, all TOTs stressed the impor- 
tance of each of the five stag^ and their interrelation; application was clearly 
an integral part of the trainiits as Tieachers were helped to plan lessons with 
such in mind. 

It was noted that over half of the training sessions observed (6 of 11) dealt 
either with familiarizing the teacher with the five stages or with helping them 
to interpret the criterion- reference test data (in the first learning stage) 
using these data as a basis for instruction, . The other five sessions were devoted 
to specific lesson planning with materials and ideas to incorporate the laboratory 
approacli. The data from teachers' interviews, which showed that all elementary 
teachers had expressed needs in content areas in mathematics, and the content of 
the above- described training sessions lead us to conclude that the TOTs are teaching 
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the necessity of meshing skill development and applications. However, the fo 
of the teachers' interest as self-perceived needs, is on more basic concerns 
(e.g., diagnosis, lesson planning, and subject matter content). 
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12. By February 1, a detailed plan which describes the process of integrating RENP's 
math "configuration/program" with the existing math programs C^CPS, Title I, for 
example) at each school will be developed. By March 1, it will be implemented. 

Criteria 

1. A detailed plan which describes the process of integrating RENP's math 
"configuration/program" with existing math programs at each school will have 
been developed. 

2. The plan will have been implemented: 

- develop the use of the. laboratory approach to the teaching of mathematics 
with the teachers' ongoing curriculum for student instruction and teacher 
training 

- coordinate services at each school with the other specialists (Title I 
and resource teachers) 

-.searve grade levels not addressed by Title I 

Methods 

Criterion (1) The submitted plan was reviewed to determine what specific actions 
were to be taken in schools. 

Criterion (2) Nineteen training sessions with teachers were observed; all TOTs, 
nine teachers, all principals and/or assistant principals, and two math resource 
teachers were interviewed. 

Has the Grant Term Been Met? 

The grant term has been fully met at high quality. 



The submitted plan contains a description of the programs and personnel 
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-at each school and details what actions the TOT will be pexformng. 

Criterion .(2) The observed training sessions indicated diat ISBiP does not have its 
own curriculum at these schools but uses the existing cmEricuItorm of each teacher in, 
the training for the laboratory approach^ The stoident plams and. teachers' lesson 
plans which are developed with the TOT -were all showm to he besed upon the D.C.P.S. 
curricular topics currently taughf by :ne teachers. The interasws with both 
teachers and TOTs confirmed thesB observations. 

Interviews with principals confirmed tie infonnatiom ccKSEine'd in the submirted" 
plan as to the programs and personnel toper^.i^ne in :t±eij; schomls. Specifically, 
RENP does not work with teaches participating in Title 1. m irrterviews with both 
principals and two math resource teachers all stated tte TOT3 regularly meet 
with the specialists to discuss ways to coordinate their service to teachers. 
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13. By February 1, almost every employee of the math component will have been 
evaluated/assessed for competency in teaching math. Incompetent personnel 
will be dismissed. By March 1, others will have received intensive 
training to bring them up to speed. The personnel records will show that 
this was done. Periodic checks to ensure competence is maintained should 
be apparent in personnel folder. TOTs, TOAs, and lAs will know what they 
are doing, their activities will make educational sense, they will know 
why they are conducting various activities, and perform them with high 
levels of competence. 



Criteria 

1. Almost every employee of the mathematics component will have b^en 
evaluated/assessed for competency in teaching mathematics. 

2. Personnel will have received training for the purpose of improvement 
in areas of deficiency. 



3. Personnel training shall have been recorded and periodically 
updated. 

4. TOTs, COA, and lAs will fully understand the activities^f their 
roles. 



S. TOTs, COA, and lAs will perform their required activities with 
high levels of competence. 



Method 

Criteria (1 and 3) Ther personnel folders of all the TOTs, COA, and lAs were 
reviewed, and the Director of the Mathematics Component was interviewed. 

Criterion (2) The training of the RENP math staff was observed in four staff 
development sessions of TOTs and COA and three training sessions of lAs, 

Criterion (43 All TOTs, COA, and lAs were interviewed to assess their under- 
standing of their roles as described in the ''Training Programs" document. 
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V Criterion (5) All TOTs were observed in 19 training sessions with teachers. 
The COA was observed. in 3 training sessions with aides. All lAs were observed 
instructing students in the labs. In addition, 9 teachers and all 10 principals 

;.■ and/or assistant principals were interviewed.! ' 

Has the Grant Term Been Met? 

All criteria have been fully met with a high degree of quality. 
Eviden ce 

i 

Criterion (1) Completed Staff Assessment Forms were observed for all TOTs, COA, 
and lAs in the Mathematics Component. 

Criteria (2 and 3) Training plans exist for all TOTs, COA, and lAs, For TOTs 
and COA these plans specifically addressed the skills measured by the various 
assessment instruments, and included a description of the area in which work was 
needed, and at least one course for remediating weaknesses. Individualized 
Training Plans existed for each of the lAs, and each Training Plan was calibrated 
to the results of the staff assessment. The Individualized Training- Plans for 
all TOTs, COA, and lAs are updated monthly, and evidence of such updating such 
as certification of skills in which a TOT, a COA, or an lA was initially judged 
deficient were immediately apparent. The observed training sessions were 
commensurate with the training plans. 

Criterion (4) A document entitled "Training Program/Mathematics Component/RENP" 
clearly specifies those activities to be perf<irmed by TOTs, COA, 'and lAs, along 
.with reasons for conducting these activities. Second, all employees in the 
Mathematics Component are required to review their monthly assessment ratings as 
evidenced by the fact that all the rating sheets had been countersigned by employees. 
Third, interviews conducted with TOTs, COA, and lAs- substantiated that they are 
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cognizant of activities they are to perform, and the reasons for conducting 
various activities. 

Criterion (S) All of the mathematics TOTs were observed training teachers on 
two occasions (excepting one TOT who was seen only once due to illness). In all 
•cases we concluded that TOTs were performing tjt high levels of competence.. For 
a detailed discussion of this latter point, the reader is referred to grant term 14. 

The performance of the COA was also judged to be at a high level of competence as 
evidenced by our visits to three training sessions the COA conducted for lAs. 

Observations of all of the lAs indicate that they are performing their jobs compe- 
tently and in accordance with their job descriptions, A further discussion of the 
quality of the performance of lAs appears in the description of grant term 9. 

Finally, the interview data from discussions with teachers and principals, indi- 
cate that the TOTs and the lAs are performing their jobs competently. 
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14. By February 1, almost all teachers (estimated 84) will have been assessed 
for their strengths/weaknesses in teaching math. An individual teacher 
learning plan will be developed, showing what activities will be undertaken 
to bring teachers close to 100% competency. By March i; most teachers will 
have been trained- With regard to their math activities in the classrooms, 
they will know what they are doing re math, why they are doing this, and 
show high levels of competency in implementing activities. 



Criteria 

1. RENP will have developed criteria for the competency in the teaching of 
mathematics which will be used as a basis for the assessment of teachers and for 
their training. The criteria should be logical, specific, and understood by the 
TOTs to provide such a basis. 

2. The assessment of teachers' skills will have been completed on almost all 
teachers and will have been performed in a standard and formal manner with instru- 
ments/procedures logically derived from the competency criteria, 

3. The training of teachers will have been done in accordance with the training 
plans at a high quality; 

theory sessions will move from theory into practice including not 
only stimulating concepts and ideas but also specific teaching 
strategies, technique and materials designed to foster instructional 
improvement in the RENP competency criteria areas. 

— demonstration lessons by TOTs will be of high quality, exemplifying 
the skills in which teachers are being trained (i.e., the competency 
criteria) and include appropriate debriefing (feedback) with teachers. 

— TOTs and teachers will engage in planning for teachers' new instruc- 
tion, fostering instructional improvement in the RENP competency 
criteria areas. ■ 
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— TOTs will observe teachers' demon stxsx±ng new skills through regular 
classroom observation. 

— TOTs will record the training/staff development activities of the 
teachers (i.e., modules completed) and instructional improvement 
demonstrated and will encourage teachers^ self-evaluation. 

4. The teachers will be aware of their instructional improvement in the teaching 
of mathematics, will demonstrate competency in the areas of their RENP training, 
and will have positive attitudes towards their continued instructional improvement. 

Method 

Criterion (1) Competency criteria which RENP requires of teachers in the person- 
alized approach to Mathematics instruction were reviewed. 

•Criterion (2) The process by which teachers* skills and/or needs were assessed 
was analyzed through the 'review of instruments, the scrutiny of all TOTs* files 
oh each of their teachers, and the examination of all TOTs* weekly reports on 
activities to the component director. 

Criterion (3) The process by which teachers were trained was analyzed through 
scrutiny of all TOTs* files on each of their teachers, through observation of 
19 training sessions (H theory/feedback/planning sessions and 8 demonstration 
lessons) involving 13 teachers (or 15% of the 85 teachers) in the ten schools. 

Criterion (4) The instructional improvement of teachers and their attitudes 
towards training were assessed through interviews with 9 teachers (or 11% of the 
85 teachers a random sample, stratified by TOT), through intervie- s with the 
principals of all ten schools, by scrutiny of TOT records on teacher training 
activities and by observation of the teaching of two classroom teachers. 
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Has the Grant Term Been Met ? 

Criterion (1) has been fully met with a high degree of quality. 

Criterion (2) has been fully met in seven of the schools, with a high degree of 
quality. The other three schools have only partially met this criterion although 

what was done showed high quality. 
•Criterion C^) has been fully met at all schools with a high degree of quality. 
Criterion (4) is judged upon limited evidence. As reflected in the formative 
evaluation plan, the assessment of teacher competence with classroom observation 
•might not be fully realized; more observations were planned and desired but only 
two could be scheduled!'' The interview and documentary data do suggest strongly 
that progress in performance and positive . changes in attitude have been affected 
by the project. The extent a£ change in teachers' performance remains in question. 

Eviden ce 

Criterion (1) The follovdjig competency cr±reria have been developed by RENP 
for the teaching of ^math3ematics: 

— teachers will be able to use diagnostic/prescriptive teaching (i.e., will 
r be able to administer and analyze diagnostic tests, construct a 

behavioral hierarchy for a given topic, teach an individual student 
or groups of students with similar needs following the five stage 
learning sequence of RENP math component, teach students with learning 
disabilities in mathematics, provide individualized instruction 
for classes) 

" teachers will be able to establish learning centers in their classrooms. 

— teachers will be able to use the laboratory approach and the discovery 
method in teaching in the stages of the learning sequence 

— teachers will be able to mesh students' skill development and 
application 

— teachers will be able to teach a topic using a variety of methods 

• u 46 
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• These criteria are logical and specific, in light of RENP's goals. All TOTs intea?- 

viewed understood that these competencies were the basis of the teacher training 
. program. 

Criterion (2) The RENP Mathematics TOTs sent Letters of Inquiry (a teacher self- 
assessment form) to all 85 teachers. The form explicitly states the competency 
■ criteria (with the exception of meshing development and application) to teachers 
and asks teachers to indicate their knowledge/performance of them on seven specific 
teaching activities. An additional seventeen content areas (including applications) 
are to be similarly rated for needs by the teachers. At seven of the ten schools 
all teachers completed this form; in the remaining three schools 5 of 6 (63%), 3 of 7 
(43%), and 12 of 16 (75%) did complete the form. 

The TOT followed up the Letter of Jnquiry with an individual conference with each 
teacher to plan fortheTOT to obsg^nre rthe teacher and his^/hor class during instruc- 
tion. After this observation they~:again conferred to develop a mutually agreeable 
set. of training areas - the specific teaching skills to be addressed in training 
and the teacher's curriculum in which he/she would be learning to teach through 
the laboratory approach and the learning stages. The .weekly reports submitted 
by the TOTs to the component head were reviewed by the formative evaluators. TTiey 
indicated all TOTs both observed and conferred with all teachers in 9 of 10 schools. 

From the above negotiations between TOT and teachers, individual profiles of 
areas of needed training were to be drawn. In evidence to the formative evaluators 
were such prodiles for all teachers in seven schools; in ,-he remaining three schools 
5 of 6 (63%), 4 of 7 (57%), and 13 of 16 (81%) were found, completed. 

On the basis of the individual teacher profiles training plans were to be developed 
which scheduled the training of each teacher at each school and indicated the nature 
of each training session. As with the teacher profiles, teacher plans were completed 
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for all the teachers at seven of the -schools and for the same percentages of 
teachers as the profiles at the remaining three schools. 

Criterion (3) The formative evaluators observed 19 teacher training/staff develop- 
ment sessions which permitted the observation of all TOTs twice with one exception 
in which illness of evaluators and TOT on several occasions allowed only one session 
to be observed. In the 11 theory/feedback/planning sessions (so designated because 
an integrated format was observed in each) the TOTs demonstrated generally high 
quality training in accordan^'i' with tke criteria: theory into practice 'and planning 
which fostered RENP criteria. In the eight demonstration lessons observed, all six 
TOTs exemplified the skills to he ts^ight and did utilize sufficient time to debrief 
with the teachexs involved. Howe^er^ we noticed definite Jmdividual variability 
among TOTs in these criteria as we IL^as variability betweerr. different sessions of 
the same TOT. We judge that while e^h of the TOTs have h&n observed to meet the 
aforementioned standards of quality, .some TOTs could imp rov.e. their consistency of 
perfoimance. 

The TOT at each of the ten schools has recorded the activities ..in which teachers 
have participated on each teacher's training plan. Weekly reports of these 
activities are sent to the component director. In five of the eleven staff 
development sessions, the TOTs specifically encouraged the teacliers to continue 
self- ass essmet^t. The nature of the interactive process observed between TOTs and 
teachers is one in which the teachers, even in the presence of the formative evaluators 
speak most candidly to the TOTs of theix- own weaknesses. We infer from such candor 
that a learning climate facilitative of self-evaluation has been established by the 
TOTs. All teachers are being asked to complete a questionnaire from the component 
head (who is meeting with teachers at each school to explain its purpose). They are 
. asked to evaluate the quality of their training, the TOT's performance, and their 
own instructional improvement. 
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Criterion (4) Nine teachers were interviewed by the formative evaluators. Teachers 
were randomly selected and stratified by school and TOT. All were most laudatory 
of the nature of the training they were receiving from their TOT; they each 
expressed in some detail specific ways in which their teaching had been improved 
by.-.the RENP sessions and displayed positive attitudes towards continued involvement 
with the TOT. Eight of the nine were elementary teachers and each stated that 
mathematics was a particularly difficult subject, to teach for them or constituted 
their weakest area of .preparation. Three of these eight specifically stated how 
reUeved they were toriave the TOT help them with their -slow" students for whose 
instruction they feltrparticularly unprepared. The single secondary teacher inter- 
viewed felt quite comfortable with her content knowledge, but felt RENP was providing 
helpful and effective strategies to assist with getting that ma:terial over to 
students. All nine teachers expressed positive attitucfe towards RENP and the 
training they were receiving from their TOTS. 

The interviews with principals and assistant principals tended to substantiate the 
positive attitudes expressed by the teachers towards their RENP training. Principals 
at nine of the ten schools stated that all their teachers were now generally positive 
(despite initial suspicion that the RENP training somehow reflected a poor ratiitg 
and TUB&d for training); the other principal stated that the majority of the 
teachers are now positive towards the RENP training. 

To determine the competencies gained by teachers from their RENP training, the 
formative evaluators scrutinized the TOTs' records of teachers' training sessions. 
The records indicate that definite learning had occurred in order for teachers to 
have completed specific training modules. Two classrooms were visited to observe 
the instruction. The teachers demonstrated a high degree of competence in the skill • 
area in which they had received training; yet as no baseline level of competence 
was known, growth cannot be inferred, only competence. ' ' 
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The :xelationship between criterion (3) and (4) must be noted here-. From obseirvations 
of the training sessions with 13 teachers, the fnxmative evaluatoxs have developed an 
inference regarding the level* of teaching skill snd content mastery of inany of the 
teachers involved. From analyzing diagnostic information and observing students in 
classrooms and centers, further inferences have been made. Specifically, RENP is 
dealing largely with upper elementary grade teachers (10 first grade teachers, 22 
secondary school teachers, and 53 fourth or fifth grade teachers) whose srtudents 
are predominately two grade levels or more behind their instructional level- Further- 
more, these elementary teachers in interviews, frequently expressed their own 
difficulties with teaching mathematics. This suggests that the TOTs are attempting 
to move many teachers from a very low level of teaching skills (in terms of the 
RENP competence criteria) to acceptable levels of teaching. Specifically, of the 
13 teachers with whom the formative evaluators have interacted, 8 would have to 
be judged from their own statements in training sessions to be at the most 
rudimentary ' levels in relation to RENP's competence cri.t6ri$^^ Only t:5iree cotijbdi. 
be judged knowledgeable in relation to these criteria. Thus, xt seejns most un- 
■ likely that high levels of competenx:y would be demonstrated by ''•mosi: teachers" 
(more than. 50%) if more than half began at a low level (8 of 15) and had only 
three months of training. However, it is quite conceivable that all could show 
. -progress. The testimony of our sample suggests most did, but the extent of 
progress remains uncertain, due to the lack of baseline data. 

An additional factor which the formative evaluators have considered in attempting 
to assess teacher competence or instructional improvement has been the amount of 
individualized training time TOTs can spend with teachers. This time is inversely 
proportional to the number of teachers they must serve. Also, related is the 
number of schools to which TOTs must travel, which partially determines their 
"availability" to teachers for informal assistance with materials or planning 
and spontaneous conferences. 50 
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-There exist :„rked differences fa, the pattern of TOT ,ssie„.e„t which we Judge 
to affect the type and the quality of the training which TOTs can give, mile 
the actual nu-hers of teachers which TOTS serve does nof vary greatly, ranging 
from 11 to 16, when partitioned by number of schools served (meaning „u„bor of 
mathematics laboratories to be established and number of days accessible to 
teachers) quite a bit of variability is evident. Specifically, two TOTs remain - 
full-time in a single school having one lab to establish and maintain with 13 and 
16 teachers to whom they are available each day (except when they must be available 
for RENP staff development,. ^re^e^TOTs split time between two schools; one TOT 
Visits one of her schools only once a week and the other two TOTs split half-time 
in each school, each servicing 15 teachers. The remaining TOT services 14 teachers 
at three. schools, l^e types of service being provided contrasting the two extremes 
(13 teachers in one school versus 14 in tl.r,e schools) .re marked; one type relegates 
the status Of the TOT to that of a weekly visitor wh6 works with selected teachers 
the other is an integral part of the daily instructional program of that school 
our interview data support this inference regarding the schedule of service. IVhen 
teachers and administrators feel :the TOT Is part of the school program with a daily 
conm,ittment, they are more coope,=ative, supportive and participative with the RENP 
program. 

-me difficulty Which split schools causes in establishing a schedule of f.equent 
training sessions in mathematics is conrpounded by having only one aide per school. 
Unlike Reading, with four aides per school, which can ..lease four teachers at a 
single time for group sessions. Mathematics has all single teacher conferences. 
This makes for good individual attention but minimized the efficiency of the 
training, especially for theory sessions and demonstration lessons. The inability 
to schedule teachers fo. more than one session each week must have an effect on 
the amount of training received and instructional improvement demonstrated. 
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Farlier, in criterion (3), we reported the observed variability in the TOTs performance 
both between TOTs and within single TOTs. The variability observed between sessions 
taught by the same TOT occurred with TOTs whose time is split between schools but not 
with the TOTs who are in single schools. r>uch observations suggest that a fragmen- 
tation of service across schools may be affecting the diagnosis of teachers' needs, 
the planning of training sessions, or the interpersonal relationships between TOT 
and teachers (the impairment of any one of which could produce a reduced quality 
in training). 
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15. By February 15, almost all students (estimated 2,520) will have had individual 
profiles prepared re their strengths/weaknesses in math. An individual learning 
plan for each student will have been prepared with the assistance of the teacher. 
The April 26th review will show that the individual learning plans have been 
implemented. The individual student records will show what activity was under- 
taken, the student's progress, and what modifications were necessary to improve 
student learning in math. 

Criteria 

1. ICRT or PNrr data on students will have been received from the school. 

2. Individual profiles for each student will be kept on file. 

3. Individual Learning Plans will have been prepared and will be kept 
on file. 

4. The plans will have been implemented and progress recorded. 
Method 

Criterion (1) The printouts of test data for students were reviewed at each 
school with the TOT. 

Criterion C2) and (3) The profiles and plans of each teacher for 10% of the 
students were reviewed at each school with the TOT. 

Criterion (4) As described in grant term 16, a random sample of about 3% of student 
plans were analyzed on two occasions for the appropriateness of activities compared 
to plans. On a third and earlier occasion a 10% sample of student plans at each 
school was reviewed to determine if recording of activities and progress was being 
performed. 

Has the Grant Term Been Met? 

Criterion (1) has been met in all schools. 

Criterion (2) has been fully met at eight schools and partially met at two schools. 
Criterion (3) has been fully met at all ten schools. 
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* Criterion (4) has been fully met at all ten schools. 
Evidence 

Criterion (1) The ten schools provided test data for the TOTs to construct profiles 
with the teachers. However, due to student transfers during the sdaool year and 
student absences, approximately 85% of the students have Has ts available at nine 
schools and only 50% available at the other. An addiitionial 10% across schools 
were tested at an inappropriate " level, resulting in either 100% mastery or 0% 
mastery. 

Criterion (2) The 10% sample of student profiles at each school revealed that at 
eight schools all of the sampled students had completed pn:ofiles. At the other 
two schools 85% (17 of 20) and 75% (15 of 20) of the sampled students had completed 
profiles. We were told by. TOTs that they had administered parts of the diagnostic 
tests (in accordance with the curriculum being addressed) to students for whom 
completed tests were unavailable. Tliis retesting probably accounts for the fact 
that we found a higher percentage of completed profiles (which address current 
, curricular areas) than scho.ol administered test data. 

Criterion (3) In our samples of 10% of the students at each school all were found 
to contain Individual Learning Plans which for the curricular area to be taught 
by . the teacher ennumerated the PMT. skills (which correspond to the D.C.P.S. objectives) 
or ICRT objectives to be addressed. Since the objectives are sequenced in a behavioral 
heirarchy in numerical patterns, the teacher and TOTT (knowing these patterns) can 
plan learning activities for their teaching or for the HAS* instruction. 

Criterion (4) All of the sampled plans in each school showed that activities were 
being accurately recorded as the students' work was in their "folders as well as a 
record of the learning activity, the objective and the date. 
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• There is not a uniform procedure by which the TOTs record the mastery o£ a given 
skill or objective when such has been demonstrated via a check test; each TOT has 
her own idiosyncratic coding procedure on either the profile or the plan to indicate 
mastery. 
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le. By February 15^ almost all children will be involved in their math programs. 
There must be evidence that most children (accelerated as well as remedial) 
are being accommodated. 



Criteria 

1. Almost all children will be involved in their math programs 

2. Most students (accelerated as well as remedial) are being accommodated: 

- learning activities in labs and classrooms provided by RENP will be 
appropriate for students' levels in given topics 

- enrichment activities for students who have demonstrated, mastery 
in given topics or skill objectives will be provided 

Method 

We observed laboratory instruction of all lAs and interviewed all TOTs.and lAs 
concerning the nature and scheduling of service to children. 

At each school on two occasions the records of students learning activities were 
analyzed through partioning a 25% random sample of students into remedial, grade 
level, and accelerated categories according to test level and co.nparing activities 
to plans for a 10% random sample in each category. 

Has the Grant Term Been Met? 

The grant term has been fully met at high quality in nine schooli: and partially 
met in one school. 

Evidence 

Criterion (1) At six of the seven elementary schools all students of each teacher 
are scheduled for service in the laboratory weekly and have been so attending. 
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At the seventh elementary school, after all students were introduced to the labora- 
tory in the fall different grade levels have been utilizing the lab differently. 
This scheduling was judged in our opinion to be haphazard and has now been corrected 
so that since early April all students regularly attend the laboratory. At the " 
three secondary schools, students are referred by their teachers weekly for 
• laboratory instruction. Thus, not all students are exposed to the laboratory 
itself for each teacher. However, the TOTs occasionally have been able to 
schedule whole classes to the .Lab so that the TOT, teacher, and lA can simultaneously 
instruct students in learning centers. We observed one such occurence.. 

Criterion (2) In mathematics there were no students who were administered tests 
above their grade level or who showed a high degree of masteiy (more than 75%) on 
.the grade level tests. From our 25% samples of students at the elementary schools 
on the two occassions, we found 48 of 370 students (13%) tested at grade level with 
•all others below grade level; and 56 of 358 (16%) tested at grade level. At the 
secondary school all sampled students were tested below grade level. The RENP program 
works with only applied mathematics students at the secondary level. As the parti- 

< 

tioned group for the grade level students was relatively small, we compared 
activities to plans of approximately 20% of them rather than the intended 10% 
sample. For the below grade level students, a 10% sample was chosen for the. 
.comparison. In all cases the activities recorded were in accordance with the 
learning plans. 

As nearly all students service could be judged remedial by the decision rule of 
below grade level in testing, the second aspect of criterion (2), (which necessitates 
considering accelerated students or those for whom enrichment in a given area, 
topic, or objective is appropriate) becomes significant. In ..each of the laboratories 
there exists ample activities' for enrichment in all areas which the evaluators 
could determine. These materials are used; on over half of the lab visits made, 
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we observed the lAs giving enrichment assignments to a few students who either 
had completed the particular tasks for the day or had already mastered the objec- 
tives for a given topic and was scheduled to the lab for enrichment. 



58 

ERIC 



17. 



Activities will focus on achieving enabling obiectives ~ u 
testing, module development, etc - unlesf rh^ Sir " baseline 

these are so vital to LhieCe refunding ^?tSa ZrZ'J'^t 
tllZT ^"P^^^i^ be o'n Sndren served Lr r""' 



• Criterion 

The enabling objectives referred to in this grant tern, were assessed in 
other grant tern, in this report. Thus, the criterion is: activities con- 
. ducted by the RENP Mathematics Component will not include baseline testing, 
module development, and parent training - unless the mathematics component head 
thinks these activities are necessary in order to achieve the refunding criteria. 

Method 

All of the mathematics labs were visited twice, all of the mathematics TOTs 
and the head of the mathematics component were interviewed throughout the 
evaluation. 

Has the Grant Term Been Met ? 
The .- criterion has been fully met. 

Evidence 

Neither baseline testing, nor parent involvement were observed during the 
evaluators- visits to the mathematics labs, and the head of the Mathematics ' 
component confirmed that these activities were not being conducted. The head of 
the Mathematics Component revealed that twelve modules were developed because 
they were deemed essential for training of teachers; we observed these modules 
being used with teachers in our visits to the mathematics labs. 
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18. RENP will relx primarily upon existing materials in carrying out the math 
component. No new curriculum will be developed in grades K-6 RENP will 
include an enclosure with the sixth quarterly report summarizing its review 
of materials; describing gaps, if any, to be filled by the "technical math" 
R^Siy^L^"'^ P^^^ P^^°^ testing and review. With NIE's and 

DCPS/RENF's mutual agreement (to be received by January IS, 1976) th-- 
program will proceed to implement its plan for technical mathematics' This 
xs, however, an option at the discretion of RENP and should be undertaken 
only if the other math activities can be fulfilled. 



Criterion 

As NIE did not approve the development of "technical math" only the following 
is considered: the Mathematics Component will use existing materials and no new 
curricula will be developed in grades K-12. 

Method 

Observations were completed in ten classrooms, all of the mathematics labs, 
•nineteen staff development sessions for teachers, and seven staff development 
sessions for RENP employees in the Mathematics Component. 



Interviews were conducted with all of the mathematics TOTs and with nine teach 



Has the Grant Term Been Met? 



ers. 



The grant term has been fully met. 



Evi dence 

No new curricula development was in evidence; mathematics lab materials from Creative 
Teaching Associates were being used in the Mathematics Component; supplementary 
ideas for teachers utilized existing materials. 

All of the interviews that were conducted substantiated the above findings. 
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19. By February 1/ individualized unit task force plans will be available for all 
Phase I schools- By March 1, the unit task force plans will be almost fully 
implemented, under the oversight of thelocal school boards of Phase I schools. 

Criteria 

1. Each plan will contain clear and precise objectives, stated in measureable 

2. >^.d, ;;vlan wili contain a section indicating the various activities which will 
be un<: rriiiken by school staff to implement the objectives stated in their 
Vni^ i 7sk Force plan. 

3. Ev^cJi plsr? -rfill contain a time line of implementation for the objectives. 

4. Each pl^.:v will ensure that those constituting the Unit Task Force will be 
drawn frc.^ several school-related constituencies. 

5. Each y^lmi will include current test data on RENP st'uclents. 

6. Each plc-m will include a section delineating the way in which the Unit 
Task Force (or designated members) will monitor the implementation of the 
overall plan. 

Method 

Criterion (1) The objectives contained in each plan were reviewed by curriculum 
specialists on the evaluation team to ascertain whether or not the objectives were 
discrete, measureable, and contained specific criteria for measurement. 

Criterion (2) Each plan was reviewed by the evaluation team to ensure that it 
contained statements relating to the "translation" of objectives into specific 
activities. Interviews were conducted with the component director in charge of 
mathematics and with the math TOTs to clarify the "translation!' process, and to 
ensure that it was, in fact, occurring. Activities observed occurring at each school 
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were compared with those delineated in the plan to determine congruence. 

Criterion C3) Each plan was reviewed to ascertain whether or not the requisite 
timeline had been included. 

Criterion C4) Each plan was carefully reviewed by the Principal Investigator to 
determine who constituted the Unit Task Force in each school. Finally, Unit Task 
Force members delineated in each plan were interviewed by the evaluators to sub- 
stantiate their participation on the Unit Task Forces, specifically, all principals 
and/or assistant principals, all TOTs, and COs. 

Criterion C5) Each plan was reviewed by the Principal Investigator to ascertain 
whether or not current testing data had been included. 

Criterion C6) Each plan was reviewed by the Principal Investigator to ensure that 
it contained a statement regarding the way in which implementation of the plans 
would be monitored by members of the Task Force. 

Has the Grant Term Been Met ? 

Criterion (1) has been met for each of the ten schools with a high degree of quality. 

Criterion C2) has been met for each of the ten schools. The degree of quality varies 
between schools, though the continuum runs from "good" to "excellent". Therefore, 
the degree of quality is judged acceptable for all schools. 

Criterion C3) has been bet, with high quality for each of the ten schools. 

Criterion (4) has been fully met at each of the schools. 

Criterion (5) has been fully met, with a high degree of quality at each of the schools 

Criterion (6) has been met for each, of the schools. The quality of the monitoring 
plan, while varying for each school, is sufficient to ensure a careful check on 
whether or not the RENP activities delineated within the plan are being carried out. 



" Evidence 

Criterion (1) Each of the Unit Task Force Plans pertaining to those schools having 
. a inath component has two sets of objectives: the first set. connnon across all plans, 
is con5,rised o£ five general reading objectives embodied by the Response to Educational 
Needs Project. One o£ these objectives pertains to specific teacher training outcomes 
in math. Two. of the objectives pertain to student outcomes, and focus upon the 
acquisition of measurable, discrete skills, which students participating in RHNP are 
expected to acquire. One objective pertains to TOTs and one pertains to Instructional 
Aides. 

In addition to the five common math objectives, each Unit Task Force plan contains 
math objectives idiosyncratic to that particular school. In all cases, these objec- 
tives relate to specific student outcomes. Three schools also listed, as part of 
their own objectives, outcomes relating to teacher training. All of the individual 
school objectives were closely related to the common RENP objectives and differed 
only in the stress which they placed upon certain aspects of those objectives. 

The number of school specific-objectives varied from as few as five, in one school, 
to as many as sixteen in another. The mean number of specific school objectives was 
seven. 

criterion (2) Ihe method(s3 to be utilised in implementing Unit Task Force objectives 
wer. common across all plans, and were included in a section entitled' "Staff Develop- 
ment. Utilizing the Diagnostic/Prescriptive Approach", n^is section, in tm^n. was 
sub-divided into three smaller sections: diagnosis, (teachers) diagnosis (students) 
and prescription/application. These subsections are briefly described below: 

- Diagnosis (teachers): This section clearly delineated the specific steps to 
be followed by each TOT in constructing individual teacher profiles. (Th 
profiles, when completed form the basis for Teacher Training in relation 
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to each of the general RENP and school specific objectives.) The assess- 
ment procedures, described in detail elsewhere in this report, consisted of 
a "Letter of Inquiry"; personal interviews with TOTs and teachers, and 
observations of individual instruction by TOTs. 

— Diagnosis (students): This section briefly describes the manner in which 

teachers will be taught to diagnose student needs in relation to the objectives 
of the Unit Task Force Plan. It focuses upon the administration of diagnostic 
tests and the use of test data in the construction of individual student 
profiles • The culmination of this section addresses the (generation of 
individualized student learning plans based on such profiles. 

--Prescription/Application: This section describes, in general, the ways 
in which teachers will be urged to apply the skills which they have 
acqiaired f.rom their interactions with the TOTs. The section stresses 
the selection of specific instructional strategies to meet the needs of 
individual students^ and the selection of curricular material appropriate to 
the remediation of specific skill deficiencies. 

While the staff development section of the Unit Task Force plans is common across 
all of the schools, different aspects of the plan are stressed at different points- 
of time in the individual schools. This is reflected in the project timeline which 
is included as part of each plan, and is corroborated by repeated observations 
within the schools. 

Criterion (3) Each of the Unit Task Force plans included a timeline delineating 
when each of the specific activities delineated in the plan would occur. The time- 
HSes encompassed five months (January to May) and stipulated end- dates for each 
aspect of the plan; for example the date by which initial diagnosis would be complete;*d, 
the date by which training would begin, etc. Three of the schools also included within 
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•their timelines dates by when specific student objectives were expected to have been 
accomplished. The amount of time allocated for specific activities was similar, 
though not identical across schools. Intervieivs with Unit Task Force members 
revealed that, in constructing the time line, the Unit Task Force considered realistic 
constraints, such as the number of teachers with whom the TOTs worked, the amounts 
of time that teachers could devote to being trained, and the baseline training needs 
of teachers. 

Criterion (4) Each of the Unit Task Forces was comprised of the following categories 
of individuals: school principal Cor his/her designee); form one to three teachers, 
a representative o£ the Washington Teachers' Union, the school Librarian, the Math 
TOT, a student, a Local Baord Member, a Reading Specialist, a Mathematics Resource 
Teacher, the Community Organizer, a Counselor, and the Instructional Aides. Our 
interviews with Unit members indicate that these constituencies are represented on the 
Unit Task Forces. 

Criterion (5) Each plan contained current (1975/1976) test data on students 
serviced by RENP. Math data were collected through the use of the PMT instrument, 
which had been approved by the Component Director and the TOTs, In all cases, data 
were stratified by grade and classes, and the number for each class was reported. 

Criterion (6) Each plan contained a brief narrative describing the way in which the 
Unit Task Force would ensure that RENP activities were being implemented. All of the 
plans stressed that designated Unit Task Force members would periodically review the 
student and teacher profiles kept by the teacher and TOTs to make sure that the 
latter were current. Each of the monitoring plans also stipulated that the TOT 
must report to the Unit Task Force each month, detailing her activities during that 
month. Finally, each of the pains contained a statement that the Unit Task Force, 
as a whole, had the right and obligation to monitor any and all activities delineated 
within the larger plan at any time. One. of : the plans stipulated that it would conduct 



-its own, informal survey of teachers and students to determine whether or not the 
latter were satisfied with the services they had been receiving from RENP, 

In all but one case, the actual monitoring plans were not specific, in that they 
did not designate particular Unit Task Force personnel as being responsible for 
any given facet of the monitoring process; neither did they stipulate how they 
would review the contents of teacher and student profiles (i,e. what criteria 
would be utilized to judge the acceptability of such profiles), IVhen we compared 
interview data from TOTs and CDs concerning the monitoring done by the principal 
with the interview data from principals concerning their knowledge of RENP 
activities in their school (specifically or the math component) remarkable congruence 
was found. Five of ten principals or designated assistant principals were definitely 
monitoring RENP quite closely. 
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20. No new curriculum development will take place in the Reading Component, 
RENP will rely upon existing materials for this component. These mav 
be adapted for use as appropriate. 



Criterion 

The reading Component will use existing materials; no new curricula will be 
developed. 

Method 

Observations were completed in 7 classrooms, in all 10 reading centers, in 24 
staff development sessions for teachers, and in 7 staff development sessions 
for personnel in the Reading Component. In addition, interviews were conducted 
with all of the reading TOTs and with 2S teachers (or 21% of the 115 teachers). 

Has the Grant Term Been Met? 

The grant term has been fully met. 

Evidence 

No new currimulum development was in evidence; TOTs were observed using approved 
kits and materials. 

■nie interviews that were conducted also verified that no new curricula have 
be developed. 
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21. RENP will use existing diagnostic instmuments in the reading component 
rather than developing new ones. Informal tests desigmed to facilitate 
day-to-day classroom work will be permitted. 

Criterion 

TOTs will use only existing diagnostic tests. 
Method 

To determine if the criterion has been met,, iiuspections of discos tic test 
data and interviews of TOTs were completed. 

Has the Grant Term Been Met? 

This grant term has been fully met. 

Evidence 

Approved tests are used by all TOTs. These tests include the following: ICRT, 
PRT, Phonics Mastery Test, Botel Word Opposite Test, Keys to Reading: Competency 
Skill Test, and pre-tests from reading kits. One TOT used the DCPS .Word Recogniti 
and Competency Test for Sequential Skills, 
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22, By February 1, almost every employee of the reading component will have been 
evaluated/assessed for competency in teaching reading. Incompetent personnel 
will be dismissed,- By March 1, others will have received intensive training 
to bring them up to speed. The personnel records will show that this was 
done. Periodic checks to ensure competence is maintained should be apparent 
in personnel folder. TOTs, TOAs, and lAs will know what, they are doing, their 
activities will make educational sense, they will know why they are conducting 
various activities, and perform them with high levels of competence. 

Criteria 

1, Almost every employee of the mathematics component will have been 
evaluated/ assessed for competency in teaching reading. 

2, Personnel will have received training for the purpose of 
improvement in areas of deficienc>^. 

3, Personnel training shall have been recorded and periodically 
updated V 

A. TOTs, TOAs, and lAs will fully understand the activities of their 
roles. .v" 

5. TOTs, TOAs, and lAs will perform their required activities with high 
levels of competence. 

Method 

Criteria CI) and (3) The personnel folders of all the TOTs, TOAs , and lAs were 
reviewed, and the Director of the Reading Component was interviewed. 

Criterion (2) The training of the RENP reading staff was observed in four staff 
development sessions of TOTs and TOAs and three training sessions of lAs. 

Criterion (4) All TOTs, TOAs, and a random sample of ten of the forty (25%) lAs 
were interviewed to assess their understanding of their roles as described in the 
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."Training Program" document. 

Criterion (S) All TOTs were observed in 24 training sessions with teachers. TTie 
TOAs were observed in 3 training sessions with aides. Twenty of the forty (50%) 
lAs were observed instructing .students in centers. In addition, 25 teachers and 
all 10 principals and/or assistant principals were interviewed. 

Has the Grant Term Been Met ? 

All criteria have been fully met with a high degree of quality. 
Evi dence 

Criterion (1) Completed Staff Assessment Forms were observed for all TOTs, TOAs, 
and for all cf the 25% sample (10 of 40) lAs in the Reading Component. 

Criteria (2) and (3) Training plans exist for all TOTs, TOAs, and lAs. For TOTs 
and TOAs these plans specifically addressed the skills measured by the various 
assessment instruments, and included a description of the area in which work was 
needed, and at least one course, for remediating weaknesses. Individualized Training 
Plans existed for each of the IAs,aid each Training Plan was calibrated to the 
results of the staff assessment. The Individualized Training Plans for all TOTs, 
TOAs, and lAs are updated monthly, and evidence of such updating sudi as certifi- 
cation of skills in which a TOT, a TOA, or an lA was initially judged deficient 
were immediately apparent. The observed training sessions were commensurate with 
the training plans. 

Criterion (4) A document entitled "Training Program/Reading Coraponent/RENP" 
clearly specifies those activities to be performed by TOTs, TOAs, and lAs,. along 
with reasons for conducting these activities. Second, all employees in the 
Reading Component are required to review their monthly assessment ratings as ' 
evidenced by the fact that all the examined rating sheets had been countersigned 

70 



ERIC 



by employees. Third, interviews conducted with TOTs, TOAs, and lAs substantiated 
that they are cognizant of activities they are to perform, and the reasons for 
conducting various activities. 

Criterion (5) All of the reading TOTs were observed training teachers on two 
occasions (excepting one TOT who was seen only once due to hospitalization). In' 
al' eases we concluded that TOTs were performing at high levels of competence. 
For a detailed discussion of this latter point, the reader is referred to grant 
term 23. 

The performance of the two TOAs was also judged to be at a high level of competence 
as evidenced by our visits to three training sessions the TOAs conducted for lAs. 

Observations of twenty lAs in schools, and of 13 lAs presenting model learning 
activities in staff development sessions, indicate that they were performing . 
-their jobs competently and in' accordance with their job descriptions. A further 
discussion of the quality of the performance of lAs appears in the writeup of 
grant term 30. ■ 

Finally the interview data from discussions with teachers and principals indicate 
that the TOTs and the lAs are performing their jobs competently. 



^w3. By -February 1, almost all teachers (estimated 171). will have been assessed 
for their strengths/weaknesses in teaching reading. An individual teacher 
learning plan will be developed, showing what activities will be undertaken 
to bring teachers close to 100% competency. By March 1, most teachers will 
have been trained. With regard to their reading activities in the classrooms, 
they will know what they are doing in reading, why they are doing this, and 
show high levels of competency in implementing activities. 



Criteria 

1. - RENP will have developed criteria for the competency in the teaching of reading 
which will be used as a basis for the assessment of teachers and for their training. 
The criteria should be logical, specific, and understood by the TOTs to provide 
such a basis. 

2. The assessment of teachers' Skills will have been completed on almost all 
teachers and will have been performed in a standard and formal manner with instru- 
ments/procedures logically derived from the competency criteria. 

3. The training of teachers will have been done in accordance with the training ' 
plans at a high quality: 

" theory sessions will move from theory into practice including not" 
only stimulating concepts and ideas but also specific teaching 
strategies, technique and materials designed to foster instructional 
improvement in the RENP competency criteria areas. 

" demonstration lessons by TOTs will be of high quality, exemplifying 
the skills in which teachers are being trained '(i. e. , the competency 
criteria) and include appropriate debriefing (feedback) with teachers. 

TOTs and teachers will engage in planning for teachers' new instruc- 
tion, fostering instructional improvement in the .RENP coippetctrcy 
criteria areas. 

TOTs will foster interactive planning and sharing among teachers. 
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'Has the Grant Term Been Met ? ' " 

Criterion (1) has been fully met with a high degree of quality. 

Criterion (2) has been fully met in eight of the schools, with a high degree of 
quality. The other two schools have only partially met this criterion although 
what was done showed high quality. 

Criterion (3) has been fully met at all schools with a high degree of quality. 

Criterion (4) is judged upon limited evidence. As reflected in the formative 
evaluation plan, the assessment of teacher' competence with classroom observation 
might not be fully realized; more observations were planned and desired but only 
four could be scheduled. The interview and documentary data do suggest strongly 
that progress. in performance and positive changes in attitude have been affected 
by the project. The extent of change in teachers performance remains in question. 

Evidence 

Criterion (1) The following competency criteria have been developed by RENP for 
*the teaching of reading; 

— teachers will be abl- to use a diagnostic/prescriptive approach in 
personalizing student instruction (i.e., will be able to administer 
and analyze diagnostic tests, plan for individual and group instruc- 
tion based on diagnostic information of students' skills through 
identification of specific objectives needed, provide necessary 
teacher prescription for individuals or groups). 

— teachers will be able to incorporate the teaching of reading skills 
into all content areas (elementary teachers) and in their single 
content area (secondary). 
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— teachers will be able to mesh students' skill development and 
functional applications 

These criteria are both logical and specific in terms of RENP's goals. All TOTs 
interviewed understood these competencies were the basis of the teacher training 
program. 

Criterion (2) The RENP Reading TOTs sent Letters of Inquiry (a teacher self-assess- 
ment form) to all 115 teachers. The first four questions of this form directly 
relate to the first teacher competency criterion of RENP (diagnostic/prescriptive 
teaching). The remaining questions relate to the other two competency criteria 
with three questions asking the teacher to describe the current instructional 
program and additional questions asking the teacher to rank preferences for 
• approaches or topics to increase his/her-effecti veness in the training sessions. 
At eight of the ten schools forms completed for all teachers were in evidence; at 
the other three schools one or two teachers' forms were not. In two schools 
10 of 12 (83%). and 7 of 10 (70%) of the teachers' completed forms were available. 

The TOT followed up the Letter of Inquiry with an individual conference with 
each of the teachers to plan for the TOT to observe the teacher. and his/her 
class during instruction. After this observation they again conferred to develop 
a mutually agreeable set of areas for training/staff development. The weekly 
reports submitted by the TOTs to the component head were reviewed by the formative 
evaluators. These reports showed that all TOTs both observed and conferred with 
■ all their teachers with the exception of one school; a TOT who v/as split between two 
schools (filling in where a TOT had resigned) had not been able to observe all 
teachers (only six of ten were observed). At this one school, all ten had been 
conferred with, however. 

Seven of the TOTs, early in their negotiations with teachers, utilized the 
diagnostic information from the teachers own classes to' help determine areas of..;- 
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